Mode FTFW¶

This mode is based on a reliable protocol that performs message tracking. Thus, the protocol can recover from message loss, re-ordering and corruption.

In this synchronization mode you may configure ResendQueueSize, CommitTimeout, PurgeTimeout, ACKWindowSize , DisableExternalCache and StartupResync.

ResendQueueSize <value>

Size of the resend queue (in objects). This is the maximum number of objects that can be stored waiting to be confirmed via acknoledgment. If you keep this value low, the daemon will have less chances to recover state-changes under message omission. On the other hand, if you keep this value high, the daemon will consume more memory to store dead objects.

Example: ResendQueueSize 131072

Default is 131072 objects.

CommitTimeout <seconds>

This parameter allows you to set an initial fixed timeout for the committed entries when this node goes from backup to primary. This mechanism provides a way to purge entries that were not recovered appropriately after the specified fixed timeout. If you set a low value, TCP entries in Established states with no traffic may hang. For example, an SSH connection without KeepAlive enabled.

Example: CommitTimeout 180

By default, this option is not set (the daemon uses an approximate timeout value calculation mechanism).

PurgeTimeout <seconds>

If the firewall replica goes from primary to backup, the `conntrackd -t command' is invoked in the script. This command schedules a flush of the table in N seconds.

This is useful to purge the connection tracking table of zombie entries and avoid clashes with old entries if you trigger several consecutive hand-overs.

Default is 60 seconds.

ACKWindowSize <value>

Set the acknowledgement window size. If you decrease this value, the number of acknowlegdments increases. More acknowledgments means more overhead as conntrackd(8) has to handle more control messages. On the other hand, if you increase this value, the resend queue gets more populated. This results in more overhead in the queue releasing.

Example: ACKWindowSize 300

If not set, default window size is 300 (value is based on some practical experiments measuring the cycles spent by the acknowledgment handling with oprofile).

DisableExternalCache <yes|no>

This clause allows you to disable the external cache. Thus, the state entries are directly injected into the kernel conntrack table. As a result, you save memory in user-space but you consume slots in the kernel conntrack table for backup state entries. Moreover, disabling the external cache means more CPU consumption. You need a Linux kernel >= 2.6.29 to use this feature.

If you are installing conntrackd(8) for first time, please read the user manual and I encourage you to consider using the fail-over scripts instead of enabling this option!

By default this is set to no, meaning the external cache is enabled.

StartupResync <yes|no>

Order conntrackd to request a complete conntrack table resync against the other node at startup. A single request will be made.

This is useful to get in sync with another node which has been running while we were down.

Example: StartupResync yes

By default, this clause is set to no.

Mode ALARM¶

This mode is spamming. It is based on a alarm-based protocol that periodically re-sends the flow state to the backup firewall replicas. This protocol consumes a lot of bandwidth but it resolves synchronization problems fast.

In this synchronization mode you may configure RefreshTime, CacheTimeout, CommitTimeout and PurgeTimeout.

RefreshTime <seconds>

If a conntrack entry is not modified in <= N seconds, then a message is broadcasted. For example, this mechanism may be used to resynchronize nodes that just joined the multicast group.

Example: RefreshTime 15

CacheTimeout <seconds>

If we don't receive a notification about the state of an entry in the external cache after N seconds, then remove it.

Example: CacheTimeout 180

CommitTimeout <seconds>

Same as in FTFW mode.

PurgeTimeout <seconds>

Same as in FTFW mode.

Mode NOTRACK¶

Is the most simple mode as it is based on a best effort replication protocol, ie. unreliable protocol. This protocol sends and receives the state information without performing any specific checking.

In this synchronization mode you may configure DisableInternalCache, DisableExternalCache, CommitTimeout, PurgeTimeout and StartupResync.

DisableInternalCache <yes|no>

This clause allows you to disable the internal cache. Thus, the synchronization messages are directly sent through the dedicated link.

This option is set to no by default.

DisableExternalCache <yes|no>

Same as in FTFW mode.

CommitTimeout <seconds>

Same as in FTFW mode.

PurgeTimeout <seconds>

Same as in FTFW mode.

StartupResync <yes|no>

Same as in FTFW mode.

MULTICAST¶

This section indicates to conntrackd(8) to use multicast as transport mechanism between nodes of the firewall cluster.

Please note you can specify more than one dedicated link. Thus, if one dedicated link fails, the daemon can fail-over to another. Note that adding more than one dedicated link does not mean that state-updates will be sent to all of them. There is only one active dedicated link at a given moment.

The Default keyword indicates that this interface will be selected as the initial dedicated link. You can have up to 4 redundant dedicated links.

Note: use different multicast groups for every redundant link.

Example:

	Multicast Default {
		IPv4_address 225.0.0.51
		Group 3781
		IPv4_interface 192.168.100.101
		Interface eth3
		SndSocketBuffer 1249280
		RcvSocketBuffer 1249280
		Checksum on
	}
	Multicast {
		IPv4_address 225.0.0.51
		Group 3782
		IPv4_interface 192.168.100.102
		Interface eth4
		SndSocketBuffer 1249280
		RcvSocketBuffer 1249280
		Checksum on
	}

IPv4_address <address>

Multicast address: The address that you use as destination in the synchronization messages. You do not have to add this IP to any of your existing interfaces.

Example: IPv4_address 255.0.0.50

Group <number>

The multicast group that identifies the cluster.

Example: Group 3780

If any doubt, do not modify this value.

IPv4_interface <address>

IP address of the interface that you are going to use to send the synchronization messages. Remember that you must use a dedicated link for the synchronization messages.

Example: IPv4_interface 192.168.100.100

Interface <name>

The name of the interface that you are going to use to send the synchronization messages.

Example: Interface eth2

SndSocketBuffer <number>

This transport protocol sender uses a buffer to enqueue the packets that are going to be transmitted. The default size of this socket buffer is available at /proc/sys/net/core/wmem_default.

This value determines the chances to have an overrun in the sender queue. The overrun results in packet loss, thus, losing state information that would have to be retransmitted. If you notice some packet loss, you may want to increase the size of the buffer. The system default size is usually around ~100 KBytes which is fairly small for busy firewalls.

Note: The NOTRACK protocol is best effort, it is really recommended to increase the buffer size.

Example: SndSocketBuffer 1249280

RcvSocketBuffer <number>

This transport protocol receiver uses a buffer to enqueue the packets that the socket is pending to handle. The default size of this socket buffer is available at /proc/sys/net/core/rmem_default.

This value determines the chances to have an overrun in the receiver queue. The overrun results in packet loss, thus, losing state information that would have to be retransmitted. If you notice some packet loss, you may want to increase the size of the buffer. The system default size is usually around ~100 KBytes which is fairly small for busy firewalls.

Note: The NOTRACK protocol is best effort, it is really recommended to increase the buffer size.

Example: RcvSocketBuffer 1249280

Checksum <yes|no>

Enable/Disable message checksumming. This is a good property to achieve fault-tolerance. In case of doubt, use it.

UDP¶

This section indicates to conntrackd(8) to use UDP as transport mechanism between nodes of the firewall cluster.

As in the Multicast configuration, you may especify several fail-over dedicated links using the Default keyword.

Example:

	UDP {
		IPv4_address 172.16.0.1
		IPv4_Destination_Address 172.16.0.2
		Port 3781
		Interface eth3
		SndSocketBuffer 1249280
		RcvSocketBuffer 1249280
		Checksum on
	}

IPv4_address <address>

UDP IPv4 address that this firewall uses to listen to events.

Example: IPv4_address 192.168.2.100

IPv6_address <address>

UDP IPv6 address that this firewall uses to listen to events.

Example: IPv6_address fe80::215:58ff:fe28:5a27

IPv4_Destination_Address <address>

Destination IPv4 UDP address that receives events, ie. the other firewall's dedicated link address.

Example: IPv4_Destination_Address 192.168.2.101

IPv6_Destionation_Address <address>

Destination IPv6 UDP address that receives events, ie. the other firewall's dedicated link address.

Example: IPv6_Destination_Address fe80::2d0:59ff:fe2a:775c

Port <number>

UDP port used

Example: Port 3780

Interface <name>

Same as in the Multicast transport protocol configuration.

SndSocketBuffer <number>

Same as in the Multicast transport protocol configuration.

RcvSocketBuffer <number>

Same as in the Multicast transport protocol configuration.

Checksum <yes|no>

Same as in the Multicast transport protocol configuration.

TCP¶

You can also use Unicast TCP to propagate events.

If you combine this transport with the NOTRACK mode, it becomes reliable.

The TCP transport protocol can be configured in exactly the same way as the UDP transport protocol.

As in the Multicast configuration, you may especify several fail-over dedicated links using the Default keyword.

Example:

	TCP {
		IPv6_address fe80::215:58ff:fe28:5a27
		IPv6_Destination_Address fe80::215:58ff:fe28:5a27
		Port 3781
		Interface eth2
		SndSocketBuffer 1249280
		RcvSocketBuffer 1249280
		Checksum yes
	}

OPTIONS¶

Other unsorted options that are related to the synchronization protocol or transport mechanism.

TCPWindowTracking <yes|no>

TCP state-entries have window tracking disabled by default, you can enable it with this option. As said, default is off. This feature requires a Linux kernel >= 2.6.36.

ExpectationSync <on|{ list }>

Set this option on if you want to enable the synchronization of expectations. You have to specify the list of helpers that you want to enable.

This feature requires a Linux kernel >= 3.5.

Example, sync all expectations:

	ExpectationSync on

Example, sync given expectations:

	ExpectationSync {
		ftp
		ras
		q.931
		h.245
		sip
	}

By default, this option is disabled.

UNIX¶

Unix socket configuration. This socket is used by conntrackd(8) to listen to external commands like `conntrackd -k' or `conntrackd -n'.

Example:

	UNIX {
		Path /var/run/conntrackd.ctl
	}

Path <filename>

Absolute path to the Unix socket.

Example: Path /var/run/conntrackd.ctl

Backlog <value>

Deprecated option.

FILTER¶

Event filtering. This clause allows you to filter certain traffic.

There are currently three filter-sets: Protocol, Address and State. The filter is attached to an action that can be: Accept or Ignore. Thus, you can define the event filtering policy of the filter-sets in positive or negative logic depending on your needs.

You can select if conntrackd(8) filters the event messages from user-space or kernel-space. The kernel-space event filtering saves some CPU cycles by avoiding the copy of the event message from kernel-space to user-space. The kernel-space event filtering is prefered, however, you require a Linux kernel >= 2.6.29 to filter from kernel-space.

The syntax for this section is: Filter From <from> { }.

If you want to select kernel-space event filtering, use the keyword Kernelspace instead of Userspace.

Example:

	Filter From Userspace {
		Protocol Accept {
			TCP
			SCTP
			DCCP
		}
		Address Ignore {
			IPv4_address 127.0.0.1
			IPv6_address ::1
		}
		State Accept {
			ESTABLISHED CLOSED TIME_WAIT CLOSE_WAIT for TCP
		}
	}

Protocol <policy> { <protocols list> }

Accept only certain protocols: You may want to replicate the state of flows depending on their layer 4 protocol.

Policy is one of Accept or Ignore.

Protocols are: TCP, SCTP, DCCP, UDP, ICMP and IPv6-ICMP.

The ICMP and IPv6-ICMP protocols require a Linux kernel >= 2.6.31.

Example:

	Protocol Accept {
		TCP
		SCTP
		DCCP
	}

Address <policy> { <addresses list> }

Ignore traffic for a certain set of IP's: Usually all the IP assigned to the firewall since local traffic must be ignored, only forwarded connections are worth to replicate.

Note that these values depends on the local IPs that are assigned to the firewall.

You may specify several IPv4_address and/or IPv6_address directives. You can also specify networks in CIDR format.

Policy is one of Accept or Ignore.

Example:

	Address Ignore {
		IPv4_address 127.0.0.1 # loopback
		IPv4_address 192.168.0.100 # virtual IP 1
		IPv4_address 192.168.1.100 # virtual IP 2
		IPv4_address 192.168.100.100 # dedicated link ip
		IPv4_address 192.168.0.0/24
		IPv6_address ::1
	}

State <policy> { <states list> for TCP }

Filter by flow state. This option introduces a trade-off in the replication: it reduces CPU consumption at the cost of having lazy backup firewall replicas.

Note: only affects TCP flows.

The existing TCP states are: SYN_SENT, SYN_RECV, ESTABLISHED, FIN_WAIT, CLOSE_WAIT, LAST_ACK, TIME_WAIT, CLOSED and LISTEN.

Policy is one of Accept or Ignore.

Example:

	State Accept {
		ESTABLISHED CLOSED TIME_WAIT CLOSE_WAIT for TCP
	}

SCHEDULER¶

Select a different scheduler for the daemon, you can select between RR and FIFO and the process priority.

Using a RT scheduler reduces the chances to overrun the Netlink buffer and conntrackd(8) uses by default RR unless FIFO is selected. See sched_setscheduler(2) for more information.

Example:

	Scheduler {
		Type FIFO
		Priority 99
	}

Type <type>

Supported values are RR or FIFO.

Default: RR

Priority <value>

Value of the scheduler priority. Minimum is 0, maximum is 99.

Default: 99 (as returned by sched_get_priority_max(2) for SCHED_RR)

STATS EXAMPLE¶

This configuration example tells conntrackd(8) to work as a stats collector.

Stats {
	LogFile yes
	NetlinkEventsReliable no
	Syslog yes
}
General {
	Systemd yes
	HashSize 8192
	HashLimit 65535
	Syslog yes
	LockFile /var/lock/conntrack.lock
	UNIX {
		Path /var/run/conntrackd.ctl
	}
	NetlinkBufferSize 262142
	NetlinkBufferSizeMaxGrowth 655355
	Filter {
		Protocol Accept {
			TCP
			UDP
		}
		Address Ignore {
			IPv4_address 127.0.0.1
			IPv6_address ::1
		}
	}
}

SYNC EXAMPLE 1¶

This example configures synchronization in FTFW mode with Multicast transport.

It includes common general configuration as well.

Note: this is one of the recommended setups for conntrackd(8) in a firewall cluster environment.

Sync {
	Mode FTFW {
		ResendQueueSize 131072
		PurgeTimeout 60
		ACKWindowSize 300
		DisableExternalCache no
	}
	Multicast {
		IPv4_address 225.0.0.50
		Group 3780
		IPv4_interface 192.168.100.100
		Interface eth2
		SndSocketBuffer 1249280
		RcvSocketBuffer 1249280
		Checksum yes
	}
	Multicast Default {
		IPv4_address 225.0.0.51
		Group 3781
		IPv4_interface 192.168.100.101
		Interface eth3
		SndSocketBuffer 1249280
		RcvSocketBuffer 1249280
		Checksum yes
	}
	Options {
		TCPWindowTracking no
		ExpectationSync yes
	}
}
General {
	Systemd yes
	HashSize 32768
	HashLimit 131072
	LogFile yes
	Syslog no
	LockFile /var/lock/conntrack.lock
	UNIX {
		Path /var/run/conntrackd.ctl
	}
	NetlinkBufferSize 2097152
	NetlinkBufferSizeMaxGrowth 8388608
	NetlinkOverrunResync yes
	NetlinkEventsReliable no
	EventIterationLimit 100
	Filter From Userspace {
		Protocol Accept {
			TCP
			SCTP
			DCCP
		}
		Address Ignore {
			IPv4_address 127.0.0.1
			IPv4_address 192.168.100.0/24
			IPv6_address ::1
		}
	}
}

SYNC EXAMPLE 2¶

This example configures synchronization in NOTRACK mode with TCP transport.

It includes common general configuration as well.

Sync {
	Mode NOTRACK {
		DisableInternalCache yes
		DisableExternalCache yes
	}
	TCP {
		IPv4_address 192.168.2.100
		IPv4_Destination_Address 192.168.2.101
		Port 3780
		Interface eth2
		SndSocketBuffer 1249280
		RcvSocketBuffer 1249280
		Checksum yes
	}
	Options {
		TCPWindowTracking no
		ExpectationSync yes
	}
}
General {
	Systemd yes
	HashSize 32768
	HashLimit 131072
	LogFile yes
	Syslog no
	LockFile /var/lock/conntrack.lock
	UNIX {
		Path /var/run/conntrackd.ctl
	}
	NetlinkBufferSize 2097152
	NetlinkBufferSizeMaxGrowth 8388608
	NetlinkOverrunResync yes
	NetlinkEventsReliable no
	EventIterationLimit 100
	Filter From Userspace {
		Protocol Accept {
			TCP
			SCTP
			DCCP
		}
		Address Ignore {
			IPv4_address 127.0.0.1
			IPv4_address 192.168.0.0/16
			IPv6_address ::1
		}
		State Accept {
			ESTABLISHED CLOSED TIME_WAIT CLOSE_WAIT for TCP
		}
	}
}